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Abstract 


We  propose  a  build-up  interior  method  for  solving  an  m  equation  n  variable  linear 
program  which  has  the  same  convergence  properties  as  their  well  known  analogues 
in  dual  affine  and  projective  forms  but  requires  less  computational  effort.  The  algo¬ 
rithm  has  three  forms,  an  affine  scaling  form,  a  projective  scaling  form,  and  an  exact 
form  (that  uses  pivot  steps).  In  this  paper,  we  present  the  first  of  these.  It  differs 
from  Dikin’s  algorithm  of  dual  affine  form  in  that  the  ellipsoid  chosen  to  generate  the 
improving  direction  A  in  dual  space  is  constructed  from  only  a  subset  of  the  dual 
constraints.  '  • 

At  the  start  of  each  major  iteration  t,  we  are  given  an  interior  iterate  y‘.  A 
selection  of  m  dual  constraints  is  made  using  an  “order-columns”  rule  as  to  which 
constraints  show  “the  most  promise”  of  being  tight  in  the  optimal  dual  solution.  An 
ellipsoid  centered  at  y‘  is  then  inscribed  in  convex  region  defined  by  these  promising 
constraints  and  an  improving  direction  A  computed  that  points  to  the  optimal  point 
y{  +  A  on  the  ellipsoid  boundary.  Minor  cycling  within  a  major  iteration  is  then 
started. 

During  a  minor  cycle,  the  constraints  selected  to  define  the  ellipsoid  centered  at  y‘ 
is  built  up  to  include  the  constraint  (whenever  there  is  one)  that  first  blocks  feasible 
movement  from  y*  to  y‘  +  A.  If  one  blocks,  it  is  used  to  augment  the  set  of  promising 
constraints  and  the  ellipsoid  is  revised;  the  improving  direction  A  is  recomputed  by 
means  of  a  rank-one  update,  and  the  minor  cycle  repeated  until  none  blocks  movement 
from  y*  to  y‘  +  A.  When  none  blocks,  the  minor  cycling  ends.  y,+  1  =  y*  +  A  initiates 
the  next  major  iteration.  Major  iterations  stop  when  an  optimum  solution  is  reached. 
We  prove  this  will  occur  in  a  finite  number  of  iterations. 
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1.  Steps  of  Dikin’s  Algorithm 

We  are  concerned  with  the  linear  program  whose  primal  form  is 
(P)  minimize  z  =  cx 

subject  to  x  €  {x  6  Rn  :  Ax  =  6,  x  >  0},  (l.l) 

where  c  G  Rn ,  A  €  Rm  x  " ,  and  0^66  Rm  are  given.  Its  dual  form  is 
(P)  maximize  z=yb 

subject  to  y  €  {y  €  RP1  :  c  —  yA  >  0}.  (1.2) 

We  denote  the  dual  slack  variables  by 

u  =  c  -  yA  >  0.  (1.3) 

When  feasible  solutions  exist  for  both  (P)  and  (P),  then  feasible  solutions  x  and  ( y ,  u) 
are  optimal  for  ( P )  and  (P)  if  and  only  if  the  vector 

Diag(u)x  =  0,  (1.4) 

see  Dantzig  [4].  Diag(u)  denotes  the  diagonal  matrix  with  diagonal  u.  We  therefore 
seek  feasible  solutions  for  (P)  and  (P)  satisfying  complementarity  conditions  (1.4). 

In  order  to  show  convergence  for  our  build-UD  method,  we  need  to  refer  to  specific 
parts  of  the  proof  for  Dikin’s  algorithm,  and  ‘ V  best  way  we  found  to  do  so  is  to 
first  give  our  own  proof  of  the  latter  in  the  next  two  sections.  Proofs  of  Dikin’s  affine 
scaling  algorithm  [5]  were  further  developed  by  Adler,  Karmarkar,  Resende  and  Veiga 
[1],  Barnes  [2],  Monma  and  Morton  [8],  and  Vanderbei,  Meketon  and  Freedman  [11], 
among  others.  We  will  present  a  set  of  assumptions  and  inductive  conditions  upon 
which  a  proof  of  convergence  can  be  based  and  show  how  these  extend  to  our  variant; 
hence  details  ot  their  results  with  modifications  provide  a  proof  that  our  build-up 
algorithm  also  converges  to  optimal  solutions  to  (P)  and  (P).  We  then  show  that  our 
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starting  rules  for  initiating  a  major  iteration  imply  convergence  in  a  finite  number  of 
iterations. 


Each  iteration  t  of  the  Dikin  algorithm  starts  with  am  interior  dual  y* ,  solves  an 
ellipsoid  subproblem  centered  at  y* : 

(f)  maximize  z=yb 

subject  to  y  €  £  =  {y  €  Rm  :  || (y  —  y*)AD~ 1 1|  <  1},  (2.0) 

where  the  Euclidean  norm  of  a  vector  v  is  denoted  by  ||t>||  and  D  =  Diag(u')  denotes 
the  diagonal  matrix  with  diagonal  u* . 

The  optimal  solution  to  (£ )  becomes  y‘+1  of  the  next  iteration.  We  will  review 
later  the  proof  that  y‘+ 1  lies  in  the  interior  of  (D).  It  is  computed  by  (2.2)  below.  At 
the  same  time,  a  solution  x‘  is  computed  by  (2.3)  which  satisfies  Ax *  =  6;  x‘  may  or 
may  not  be  a  feasible  solution  to  ( P ).  As  t  — *  oo,  we  will  show  that  x*  tends  towards 
being  feasible  and  also  becomes  more  and  more  complementary  to: 


u*  =  c  —  y‘ A  >  0.  (2.1) 

The  algorithm  starts  with  an  interior  solution  y1  given.  The  iterates  are  then  computed 
by 

yt+1  =y‘  +  (A6)*1'3A,  (2.2) 

x*  =  D~ 3  At  Ar  (2.3) 

where 

A  =  bT  (AD~3  AT)~l  and  D  =  Diag(u*).  (2.4) 

The  superscript  r  denotes  transpose. 

In  place  of  (2.2),  an  adjusted  yt+l  is  often  used  that  makes  a  bigger  step: 

adj.  y*+ 1  =  y*  +  a(A6)~ 1/3  A,  a  >  1, 
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where  a  is  chosen  so  that  adj.  y*+ 1  is  .9  of  the  way  in  the  direction  A  from  unadjusted 
yt  + 1  given  by  (2.2)  to  the  boundary  of  (P).  The  proof  of  convergence  is  almost  exactly 
the  same.  For  our  development,  we  assume  a  =  1. 

2.  Motivation  for  the  Build-Up  Algorithm 

The  improved  point  y,  +  1  =  y*  +  A  =  y*  +  (A6)_1/2A  results  from  solving  the 
ellipsoid  constrained  subproblem  (2.0).  The  computation  of  bT  (AD~  3  AT )~ 1  involves, 
however,  all  the  columns  of  A  even  though,  in  practical  problems,  most  columns  have 
little  effect  on  the  shape  and  size  of  the  ellipsoid  and  hence  on  the  location  of  the 
optimal  point  y  =  yt+1.  Still  other  columns  may  affect  its  location  but  may  do  so 
adversely.  This  suggests  that  a  good  part  of  the  computational  work  could  be  bypassed 
if  one  knew  (or  had  a  good  guess  about)  which  columns  to  drop  temporarily  on  a  given 
iteration  as  non-promising.  The  ellipsoid  based  on  fewer  columns  contains  i ,  and  hence 
its  use  accelerates  convergence.  Another  motivation  connected  with  the  first  step  of 
the  build-up  algorithm  is  a  rule  for  selecting  a  promising  set  of  basic  columns;  we 
believe  it  will  speed  up  convergence  of  some  practical  problems,  especially  those  with 
a  large  number  of  columns.  Several  versions  of  the  rule  lead  to  finite  convergence,  see 
Theorem  7. 

Zikan  and  Cottle  [13]  propose  the  box  method  to  select  “likely”  columns  to  keep 
in.  Given  an  interior  point  y‘  in  (P),  they  choose  m  columns  from  A  corresponding  to 
the  m  closest  hyperplanes  in  ( P )  to  form  a  “box”  or  parallelepiped  around  y* .  Then 
the  moving  direction  is  generated  by  maximizing  yb  subject  to  the  box  constraints. 
This  approach  uses  a  subset  of  the  columns  and  replaces  the  ellipsoid  by  a  box. 

Another  approach,  the  build-down  scheme,  proposed  by  one  of  the  authors,  [12], 
is  also  designed  to  reduce  the  computational  burden.  The  algorithm  begins  with  a 
full  set  of  columns.  Then  an  eliminating  criterion  is  applied  which  identifies  columns 
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guaranteed  to  never  be  basic  in  any  optimal  solution.  Via  this  criterion,  A  is  eventually 
built-down  to  the  optimal  basis  (when  it  is  unique)  and  does  so  in  polynomial  time. 

The  main  idea  in  this  paper,  in  contrast  to  the  build-down  scheme,  is  to  present 
a  build-up  interior  method  by  selecting  from  A ,  in  each  major  iteration,  a  subset  of 
hyperplanes  (columns)  that  the  current  iterate  would  be  moving  in  some  sense  towards 
if  the  proposed  move  from  y*  in  the  direction  A  were  actually  made.  We  refer  to  the 
corresponding  columns  as  “promising”.  Thus,  during  a  major  iteration  we  work  with 
fewer  dual  constraints,  hence  less  computational  effort  per  major  iteration,  including 
the  effort  to  do  the  rank-one  update.  We  present  an  affine  scaling  variant  of  the 
algorithm  here. 

As  in  Dikin’s  method,  the  affine  scaling  variant  uses  ellipsoids  in  the  subopti¬ 
mization  problem,  but  the  ellipsoid  is  modified  by  replacing  A  with  Afi ,  a  subset  of 
“promising”  columns  selected  from  A,  which  are  built  up  during  the  minor  cycling  by 
blocking  dual  constraints.  Our  goal  is  to  compare  this  variant  with  the  affine  scaling 
method  of  Dikin  [5]  (in  dual  form).  We  have  also  looked  into  the  analogous  variant  for 
the  related  Karmarkar’s  projective  algorithm  [7]  and  plan  to  make  that  the  subject  of 
a  subsequent  paper. 

The  well  known  theoretical  result  is  that  the  iterates  of  Dikin’s  algorithm  (y‘,  x') 
converge  as  t  — ►  oo  to  (y*  ,2"),  the  optimal  dual  and  primal  feasible  solutions.  Proofs 
of  convergence  can  be  found  in  [l][2][5][8][ll]  and  in  Vanderbei  and  Lagarias  [10]  under 
somewhat  weaker  assumptions.  The  convergence  ratio  p*  =  (y*b  —  yt+lb)/(y‘b  —  y*  6) 
is  asymptotically  bounded  above  by  1  —  1  /y/rh  as  t  — ►  oo.  A  proof  of  this  asymptotic 
ratio  of  decrease  cam  be  found  in  this  paper  and  in  Damtzig  [3]. 
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3.  Proof  of  Convergence  of  Dikin’s  Algorithm 
Assumptions: 

(AO)  6  ^  0,  c  ^  0,  n  >  m,  and  every  subset  of  m  columns  from  A  has  rank  m. 

(Al)  An  interior  feasible  dual  solution  y1  is  given. 

(A2)  A  feasible  primal  solution  exists. 

(A3)  Every  feasible  dual  basic  solution  is  nondegenerate. 

(A4)  Every  primal  basic  solution,  feasible  or  not,  is  nondegenerate. 

The  assumptions  imply  the  primal  feasible  region  is  bounded,  but  no  assumption  is 
made  about  boundedness  of  the  dual  space  here. 

We  must  show  first  that  the  detailed  steps  of  the  algorithm,  (2.2),...,(2.4)  are  legal 


and  hence  can  be  executed  iteratively,  namely: 

(i)  D~l  exists;  (3.1) 

(ii)  (AD~3  AT)~l  exists;  (3.2) 

(iii)  Ab  >  0;  (3.3) 

(iv)  A  =  (A b)~  1/2  A  is  an  improving  direction,  i.e.,  yt+1b  >  y‘6;  (3.4) 

(v)  u‘+1  =  u‘  -  ||Dx*||-1(£>3z‘)r.  (3.5) 

(vi)  0  <  u‘+1  <  2u‘.  (3.6) 


Proof.  We  are  given  inductively  by  (2.1)  that  y*  is  an  interior  point  meaning  u‘  = 
c  —  y*  A  >  0.  Thus,  D  =  Diag(u‘)  has  a  positive  diagonal  and  therefore  (i)  D -1  and 
D~ 3  exist  and  have  positive  diagonals.  The  rank  of  A  by  (AO)  is  m  and  so  the  rank 
of  AD'1  (which  rescales  the  columns  j  of  A  by  (l/u‘)  >  0)  is  also  of  rank  m;  hence 
AD~7  AT  is  an  m  by  m  symmetric  positive  definite  matrix  of  rank  m,  implying  (ii), 
(AD~3AT)~l  exists. 

The  iterates  satisfy  certain  important  relationships.  First 

Ax*  =  b,  (4.1) 
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which  is  obtained  by  substituting  the  expression  for  A  from  (2.4)  into  (2.3)  and  mul¬ 
tiplying  by  A  thus: 

Ax*  =  {AD~3AT){AD-2AT)-lb  =  b  (4.2) 

which  are  valid  steps  since  D~l  and  ( AD~2AT)~l  exist.  Note  that 

Dx *  =  (tt‘1x‘,u‘i‘,...,u*ni^)T  ^  0,  ||Dz*||  >  0,  (4.3) 

because  u*  >  0  and  #  0  (since  x  =  0  in  (4.2)  would  imply  6  =  0  contrary  to  (AO)). 
Also  note  that 


0  <  ||Dx‘||2  =  {x*)r  iPx*  =  (AAD~2)D2  (D~2At  At) 

=  A{AD~2At)At  =  A{AD-2AT){AD~2ATy1b  =  A6.  (4.4) 

Thus,  (iii)  is  true.  This  implies  that  it  is  legal  to  use  (A6)_  1/2  as  a  factor  in  (3.4)  and 
(2.2)  since  (A6)  >  0.  Multiplying  (2.2)  on  the  right  by  6  and  noting  (4.4) 

y*+16  -  y*6  =  (A6)1/2  =  |j£?x‘||  >  0.  (4.5) 

Thus  (iv)  is  true;  that  is,  A  =  (A6)-1/2A  is  a  strictly  improving  direction. 

By  multiplying  (2.2)  on  the  right  by  A  and  substituting  c  -  u*  and  c  -  u‘+1  for 
y‘  A  and  y‘+1  A,  we  have  from  (2.1): 

u‘+1  =  u*  -  (A6)_1/2AA  (4.6) 

=  u*  -  j|Dx,||-lAA  (4.7) 

=  u*  -  \\Dxt\\-l{D2xt)T  (4.8) 

where  (4.7)  follows  from  (4.4);  and  (4.8)  follows  by  multiplying  (2.3)  by  D2 .  Hence 
(v)  is  true. 

We  now  verify  (vi)  that  0  <  u,+  1  <  2u*  by  showing  first  0  <  u*+1  <  2u*  and  next 
that  the  feasible  region  of  (£)  is  contained  in  the  interior  of  the  feasible  region  of  (D). 
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Let  y  be  any  point  that  lies  in  the  ellipsoid  £  and  let  u  =  c  —  yA.  An  example  of  such 


a  point  is  y  =  y*+  1  and  u  =  u*+ 1  =  c  —  y‘  +  1  A.  Then 


that  is 


This  implies  for  each  j: 


h  K)1  - 

yt".-"!)’  <t 

h  «>’  - ' 


(6>  -  u*  )3  <  . 

K-)a  “ 

-U*  <  tty  ~  U*  <tt‘, 

0  <  tty  <  2 U*. 

In  particular,  y  =  y*  +  1  is  in  £  and  therefore  (5.5)  implies 


0  <  u*+1  =  cy  -  yt+lA  j  <  2u* , 


(5.1) 

(5.2) 

(5.3) 

(5.4) 

(5.5) 

(5.6) 


What  remains  to  show  is  u*+1  >0.  Assume  on  the  contrary,  that  yt+1  is  on  the 
boundary  of  the  dual  feasible  region,  then  there  exists  a  j  =  s  such  that 


0  =  u:+1  =c.  -yt+lA,.  (6.1) 

But  equality  holding  for  (6.1)  for  j  =  s,  implies  the  same  for  (5.3)  for  j  =  s  and  this 
in  turn  implies  that  all  terms  of  the  sum  (5.2)  must  vanish  except  term  j  =  s.  Hence 
u‘+l  =  u‘  for  all  j  except  s.  Recalling  (2.1)  and  (2.2): 

0  =  u*  —  u'+ 1  =  (y‘+ 1  -  y')A,  =  (A6)-"JA/1,  j*,.  (6.2) 

Therefore,  since  A6  >  0,  by  (3.3), 

A A.j  =  0  for  all  j  #  s,  (6.3) 
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and  so  from  (2.3) 


i*  =  0  for  all  j  ^  s.  (6.4) 

However  choosing  B  to  be  any  basic  set  of  m  indices  j  which  includes  j  =  s,  we  have 

b  =  Ax*  =  Abx*b  =  A  ,  x\ , 

which  implies  that  x*  is  a  degenerate  primal  basic  solution,  contradicting  (A4).  There¬ 
fore  (vi)  is  true.  Q.E.D. 

This  completes  the  proof  that  the  detailed  steps  of  Dikin’s  algorithm  are  legal. 
The  property  that  u‘+1  <  2u*  is  not  needed  for  iteratively  executing  the  algorithm 
but  will  be  used  in  the  proof  of  convergence  which  we  now  present  in  the  form  of 
several  theorems  and  lemmas. 

But  first  some  definitions:  A  primal  solution  is  any  x  satisfying  Ax  =  b.  A  dual 
solution  is  any  (y,u)  satisfying  yA  +  u  =  c.  A  primai  solution  is  feasible  if  x  >  0;  a 
dual  solution  is  feasible  if  yA  <  c  or  u  >  0.  Neither  primal  or  dual  solutions  need 
be  feasible;  however,  all  dual  solutions  we  will  be  considering  will  be  feasible  ones.  A 
partition  of  indices  j  =  1, 2, ...,  n  into  two  sets  consisting  of  m  indices  and  the  remaining 
n  —  m  indices  will  be  denoted  by  (J9,  N).  The  set  AB  of  columns  corresponding  to  B  is 
non-singular  by  assumption  (AO)  and  hence  forms  a  basis  in  the  space  of  the  columns 
of  A. 

The  basic  primal  and  dual  solutions  associated  with  some  partition  ( B ,  N)  will  be 
denoted  by  I  and  (y, fi).  By  definition,  the  primal  solution  £  =  (xB,£s)  associated 
with  ( B,N )  is  basic ,  if  £N  =  0.  The  nondegeneracy  assumption  (A4)  also  states  that 

|i,  |  >  0  for  j  6  B.  (7.1) 

By  definition,  the  dual  basic  solution  complementary  to  £  is  (y,u)  if  yAB  =  cB  or 
Ug  =  0.  The  non-degeneracy  assumption  (A3)  asserts  that  the  dual  ba^ic  solution 
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satisfies  |uy|  >  0  for  j  £  N\  in  particular,  if  the  dual  basic  solution  is  feasible ,  then 


*n  >  0.  (7.2) 

Primal-dual  solutions  are  called  complementary  if  xyu}  =  0  for  j  =  1,2 
Theorem  1.  The  primal-dual  iterates  {u*,x‘}  tend  towards  complementarity,  i.e.,  for 
j  =  1,2, ...,  n 

u‘x*  — ►  0  as  t  —>  oo.  (8.0) 

Proof  of  Theorem  1.  We  note  that  y‘6  is  strictly  monotonically  increasing  by  (3.4) 
and  has  a  finite  upper  bound  because  by  (A2)  primal  feasible  solutions  exist.  Therefore 
from  (4.5)  we  have 

yt+1b  -  y*b  =  \\Dxt\\  — ►  0  as  t  — ►  oo.  (8.1) 

where  ||Dx‘||3  =  £(u‘x*)3  from  (4.3).  Q.E.D. 

Note  that  the  proofs  of  convergence  given  in  Theorems  2  and  3  that  follows  do 
not  require  the  dual  feasible  region  to  be  bounded. 

Theorem  2.  Given  5P  any  infinite  subsequence  of  t  =  {1,2,  ...,oo},  there  exists  a 
subsequence  S.  C  S0  and  a  partition  ( B,N )  such  that  x*  =  (xg,x^)  tends  to  the 
primal  basic  solution  (iB  =0)  and  (ySu^  ,uj, )  tends  to  the  complementary  dual 
basic  feasible  solution  (y,u B  =  O.fijv  >  0)  using  t  in  5. ,  i.e.,  as  t  — >  oo  when  t  is 
restricted  to  successive  t  in  the  subsequence  S . . 

It  is  convenient  to  break  up  the  proof  of  the  theorem  into  three  lemmas. 

Lemma  2.1.  Given  50,  any  infinite  subsequence  of  t  =  {1,2,  ...,oo},  there  exists  a 
subsequence  S.  C  50  and  a  partition  {B,N)  such  that  either,  for  j  €  N ,  u‘  >  t.  >  0 
for  all  t  in  S. ,  or,  for  j  €  B,  u*  — ►  0  using  t  in  5. . 

Proof  of  Lemma  2.1.  Define  inductively  for  j  =  1,2,  ...,n  the  nested  set  of  infinite 
subsequences  S0  D  Si  D  ...  D  Sn  as  follows: 
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initialize  j  :=  1;  S0  given;  e.  :=  oo; 
while  j  <_  n  do  begin 

£,  :=  inf(u‘)  using  t  in  Si_1\  (note  e,  >  0) 
if  6,  >  0  then  5,  :=  Sy _  r ; 

if  £y  =  0  then  Sy  :=  any  infinite  subsequence  of  Sy_x  such  that 
u‘  — ►  0  using  t  in  this  subsequence; 

j  :=  j  +  l; 

end  while; 

S.  :=  Sn\  e.  :=  min(fy  such  that  £y  >  0). 

Let  ffi  be  the  number  of  indices  j  such  that  u*  — *•  0  using  t  in  5. .  For  the  remaining 
n  -  m  indices  j,  u*  >  e.  >  0  for  all  t  in  S. .  We  have  two  possibilities: 

Case  1:  Either  fh  <  m,  in  which  case  there  exists  a  subset  N  of  n  -  m  indices  j  such 
that  for  j  in  N,  u‘  >  c.  for  all  t  in  S.\  the  partition  (B,N)  is  then  defined  by  letting 
B  denote  the  remaining  indices  j. 

Case  2:  Or  the  alternative  m  >  m,  in  which  case  there  exists  a  subset  B  of  m  indices 
such  that  for  j  G  B,  u*  — ►  0  using  t  in  5. .  The  partition  (B,  N)  is  then  defined  by 
letting  N  denote  the  remaining  indices  j.  Q.E.D. 

This  completes  the  proof  of  Lemma  2.1.  The  two  lemmas  that  follow  establish 
that  fh  =  m  and  iterates  (y* ,  z‘)  converge  using  t  in  S. .  The  complementary  basic 
solutions  associated  with  ( B,N )  are  denoted  by  (f,y). 

Lemma  2.2.  If  Lemma  2.1  resulted  in  Case  1,  then  Theorem  2  is  true. 

Proof  of  Lemma  2.2.  Under  Case  1  there  is  a  partition  (B,N)  such  that  for  j  G  N 
that  u‘  >  e.  >  0  for  all  t  in  S. .  Since  (x‘u‘ )  — ►  0  as  t  — ►  oo  by  Theorem  1  and  u‘  > 
c.  >  0  for  j  G  N  and  all  t  in  5. ,  it  follows  that  — ►  0  =  £N  using  t  in  S. ,  and  it  also 
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follows  that  Xg  —  Ag1  (b  —  AN  x*N)  —>  Ag1  b  =  iB  using  t  in  S. .  By  the  nondegeneracy 
assumption  (A4),  |2yj  >  0  for  j  6  B.  Again  noting  (x* u‘ )  — *  0  as  t  — *•  oo,  it  follows 
using  t  in  S.  that  u*B  -*  0  =  uB .  Finally,  we  have  yf  =  ( cB  -  )A~  1  -*  cBABl  =  y 

using  t  in  5.  and  hence  u*N  =  cN  —  y*  A*  —*  cN  —  yAN  =  u*  using  t  in  5. .  Moreover, 
since  u*N  >  0,  u*N  —*  uN  >  0.  By  the  nondegeneracy  assumption  (A3)  or  (7.2),  >  0 

implies  >  0. 

Hence  using  t  in  S. ,  x*  — +  x,  the  basic  primal  solution  with  respect  to  ( B ,  TV),  and 
iterates  (y‘,u*)  converge  for  t  in  S.  to  the  basic  dual  feasible  solution 
(i/>uB  =  0,u*  >  0).  Thus  Theorem  2  is  proved  for  Case  1.  Q.E.D. 

Lemma  2.3.  If  Lemma  2.1  resulted  in  Case  2,  then  Theorem  2  is  true. 

Proof  of  Lemma  2.3.  Under  Case  2,  there  is  a  partition  ( B ,  N)  such  that,  for  j  €  B, 
tL*g  — ►  0  using  t  in  S'..  As  in  final  part  of  proof  of  Lemma  2.2,  y‘  =  (cfl  -  Ug  )A~  1  — ► 
cB  Agl  =  y  using  t  in  5..  Moreover,  since  >  0,  -*  uN  >  0.  By  (A3)  or  (7.2), 

Hn  >  0  implies  uN  >  0.  Thus  — ►  uN  >0  using  t  in  5. .  But  uj,  >  0  and  uN  >  0 

imply  for  j  €  TV  there  exists  an  c.  >  0  such  that  u*N  >  e.  >  0  for  all  t  in  S. .  But  these 
were  the  conditions  we  assumed  in  the  previous  lemma,  hence  its  conclusions  apply, 
proving  Theorem  2  for  Case  2  as  well.  Q.E.D. 

Theorem  3.  As  t  — ►  oo,  (x*,y‘)  converge  to  primal  basic  and  complementary  dual 
basic  feasible  solutions  (l*,y*). 

Proof  of  Theorem  3.  In  Theorem  2,  choose  S0  =  {1,2,  ...,n}  and  let  T.  =  S.  be 
the  subsequence  obtained  for  this  choice  of  S0 ,  and  let  (2* ,  y* )  be  the  basic  primal 
solution  and  complementary  basic  dual  feasible  solution  that  (x‘ ,  y‘ )  converge  to  using 
t  in  T,.  Denote  by  (Sp,yp)  for  p  =  1,2 ,...,?  the  finite  set  of  q  other  basic  primal  and 
complementary  basic  dual  feasible  solutions.  Let  be  the  shortest  distance  between 
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any  two  extreme  points  y;  i.e., 


=  (1/4)  min  ||y*  -  y'H,  i  ?  j. 


(9.1) 


where  i ,j  =  *,1,2 

Given  any  6  >  0  and  any  extreme  points  y,  denote  by  Ns  (y)  the  ^-neighborhood 
of  y;  i.e.,  the  ball 

Ns  (y)  =  {y :  ||y  -  y||  <  *}•  (9-2) 

It  is  not  difficult  to  show  that  the  ^-neighborhoods  of  y*  and  y*>  for  p  =  1,2, ...,  q  have 
no  points  in  common.  To  complete  the  proof  we  will  need  three  lemmas. 

Lemma  3.1.  Given  any  6,  0  <  6  <  60,  the  count  of  y*  not  in  the  interior  of  any  of 
the  balls  Nt(ym)  and  Nf(yp)  for  p=  1,2,  ...,q  is  finite. 

Proof  of  Lemma  3.1.  An  iterate  y*  is  said  to  lie  outside  the  non-overlapping  balls 
if  ||y‘  -  y*||  >  6  and  ||y‘  -  yp||  >  6  for  p  —  1,2,  ...,g.  Let  Y0  be  the  subsequence  of 
all  such  y*  and  let  S0  be  the  corresponding  subsequence  of  t.  If,  on  the  contrary,  the 
count  of  t  in  S0  is  infinite,  then  by  Theorem  2  there  would  be  an  infinite  subsequence 
5.  such  that  y*  -+  y  using  t  in  5.,  implying  an  infinity  of  t  in  S0  whose  y*  are  in  a 
ball  ||y  —  yp||  <  6  for  some  p  in  {*,1,2,  contrary  to  assumption. 

Lemma  3.2.  If  the  count  of  y(  in  the  ball  Nfo  (y)  for  some  extreme  point  y  is  infinite, 
then  y‘  — >  y  using  y*  in  the  ball. 

Proof  of  Lemma  3.2.  Note  that  the  set  of  y‘  lying  in  the  ball  NSo  (y)  for  some 
extreme  y,  but  outside  the  smaller  concentric  ball  Nf  (5),  is  a  subsequence  of  Y0  defined 
in  Lemma  3.1  and  therefore  must  also  be  finite  whatever  be  6  <  So.  It  follows  that 
if  the  count  of  y‘  in  Nfa  (y)  is  infinite,  then  given  any  6,  0  <  6  <  60,  there  exists  a 
tt  such  that  for  all  t  >  tt  all  y‘  in  Nto  (5)  are  also  all  in  Nt  (y).  By  definition,  this  is 
what  we  mean  when  we  say  y*  — *  fj  using  y‘  in  N»0  (y). 
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Lemma  3.3.  Either  y*  — ►  y*  as  t  — ►  oo  or  there  exists  an  infinite  subsequence 
Ti  =  and  a  successor  subsequence  T2  =  {t,  +  l,ta  +  1,...}  such  that 

y‘  — *■  y*  for  t  =  tk  in  Tx  and  y‘  for  t  =  tfc  +  1  in  T2  for  some  p  =  p0 . 

Proof  of  Lemma  3.3.  Assume  the  latter  case  that  y‘  — ♦  y*  is  not  true.  Generate 
infinite  subsequences  T[  =  {tfc}  and  T. 2  =  {tk  +  1}  and  {pfc}  as  follows: 

initialize  k  :=  1;  Sj  :=  first  t  such  that  y*  6  NSg  (y*); 

cycle: 

tk  :=  first  t  >  sk  such  that  y*  6  iV4o  (y*)  and  y*+1  €  iV4o  (yp)  for  some  p; 

Pk  :=  Pi 

sfc  :=  first  t  >  tk  such  that  y‘  €  iV4o  (y*); 

k  =  +  1; 

repeat  cycle; 

Note  that  sk  always  exists  since  y*  —*■  y*  using  t  in  T. .  And  also  exists,  for  else 
Sfc,3fc  +  l,sfc  +2, . . .  would  all  belong  to  N(g  (y*),  implying  by  Lemma  3.1  that  y*  -»  y* 
as  t  — »  oo,  contrary  to  the  contrary  assumption.  Therefore,  since  the  subsequence 
{pfc  }  is  infinite  and  there  are  at  most  q  different  values  that  the  pk  can  assume,  there 
exists  a  p  =  po  such  that  there  is  an  infinite  subsequence  of  t  €  T[  whose  y‘  are  in 
NSa  (y* )  and  whose  successor  subsequence  yt+l  are  in  Nto  (y"° ).  Hence,  there  exists  a 
subsequence  Tx  C  T[  of  tk  and  successor  sequence  T2  c  T'2  of  tk  + 1  such  that  y‘  — *  y* 
using  t  in  7\  and  y*+l  — ►  yPo  using  t  in  T2. 

Proof  of  Theorem  3  continued.  Let  (B,JV)  be  the  basic  partition  associated 
with  y,  u*  =  c  —  y*  A  and  let  (H,  $)  be  the  basic  partition  associated  with  y° , 
uPo  =  c  -  y°  A  where  po  is  as  defined  in  Lemma  3.3  under  the  contrary  assumption 
that  y‘  does  not  converge.  Since  B  ^  B,  let  j  =  r  be  in  B  and  not  in  B.  Then  since 
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for  tk  in  Tx ,  y*‘  — ►  y*  as  k  — ►  oo:  . 


u‘r*  =  Cr  -  y'*  A.r  —  cr  -  y'A.r  =  ur*  =  0,  (10.1) 

tt‘/  +  1  =  Cr  -  y‘‘  +  14r  -  Cr  -  tf'A.r  =  tt?°  >  0,  (10.2) 

where  uPo  >  0  because  r  is  non-basic  with  respect  to  (H,  W)  and  the  dual  basic  solution 
by  (A3)  is  nondegenerate.  According  to  (3.6),  u‘+1  <  2u*.  Since  u‘‘  — ♦  0,  it  follows 
that  u‘*  +  1  — *■  0,  contrary  to  (10.2).  Therefore  the  contrary  hypothesis  of  Lemma  3.3 
holds,  namely  (x* ,  y‘)  do  converge  to  a  primal  basic  solution  x*  and  the  complementary 
dual  basic  feasible  solution  y* .  Q.G.D. 

Essentially  the  same  proof  of  Theorem  3  was  given  by  Todd  [10].  We  gave  the 
proof  above  to  show  later  where  to  modify  it  for  our  build-up  algorithm. 


Theorem  4.  Dikin’s  algorithm  converges  to  optimal  basic  primal  and  dual  solutions. 
Proof  of  Theorem  4.  It  is  clear  that  if  the  primal  x*  -*■  x*  which  is  feasible,  then 
the  complementary  conditions  (1.4)  are  satisfied,  implying  convergence  in  the  limit  to 
optimal  solutions  to  the  primal  and  dual  problems.  Since  Ax*  =  6,  we  only  need  to 
prove  that  x*  — ►  x*  >0  for  all  j.  At  the  fth  iteration,  we  have  from  (4.8) 


(u‘.)3z* 

«!+1=u‘-U4rf  =«!(!- 


\Dx*\ 


)• 


(11.1) 


WDx^l 

Let  us  assume  on  the  contrary  for  some  basic  index  r,  that  x*  — ►  x*  <  0  as  t  — *  oo. 
By  complementarity,  see  Theorem  1,  «*-►(*;=  0.  There  exists  a  finite  t  such  that 
for  all  t  >  t,  x\  <0  and  therefore,  since  u*  >  0: 


1  -  ttyX*/||£?x‘ ||  >  1  for  all  t  >  t. 


Hence,  from  (11.1),  for  all  t  >  t 


,«+i 


>  ul 


(11.2) 


(H-3) 
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Thus,  we  have  u*  strictly  increasing  for  t  =  f  4-  l,t  +  2,  ....,oo,  contradicting  u‘  — ►  0 
for  basic  index  r.  This  completes  the  proof  of  Theorem  4.  Q.E.D. 


Theorem  5.  The  ratio  of  convergence 

_(i  +  „ 

y*  b  -  y*  b 


'  m 


1/3  (12.1) 

where  — *•  0  as  t  — *  oo,  i.e.,  p*  is  asymptotically  <  1  —  (l/m)l/ra. 

Proof  of  Theorem  5.  Let  ( B,N )  be  the  partition  associated  with  optimal  (x*,y’) 
and  let  B  =  }•  Since  y*  =  cgA'1  and  y*  =  (ca  -u*B)ABl: 


y'b-y'b  =  u*BABlb  =  u*BxB 


From  (12.1)  and  (4.5), 

.*  = ,  yt+lfe~y^  _ ,  _ 

y'b  -  y*b 

We  may  rewrite  (12.3)  as: 


•in 


(12.2) 


(12.3) 


/  " 

(«;n)(i-^)=  £(«;*;)’ 


i 

m 


1/2 


1/2 


(12.4) 


1/2 


\«=  i  > 

>  (£«*;, )a + Ew.*;.)1  («/*;,)’  - 1)  I  •  (12.5) 

Define  A<  =  A*  =  u*. /uB  TB .  Note  Y17  =  1,  0  <  A<  <1,  because  u‘fl  xB  = 

u*.f*  and  tt*.**(  >  0.  Substituting  u‘.  **.  =  \ (u‘B  ia )  on  the  RHS  of  (12.5)  and 

cancelling  the  common  factor  uBiB  >0  from  both  sides: 

(m  m  \  *  /* 

•  (12.6) 

i=  1  »=1  / 

>  (1/m  +  c‘)1/3  (12.7) 


where  the  first  term  of  (12.6),  £(A<)3  ^  ^/m  since  £A<  =  1  and  the  second  term 
denoted  e*  tends  to  0  because  x*(  — ►  I*  >0  and  the  weights  0  <  A,  <  1.  This 
completes  the  proof  of  Theorem  5.  Q.E.D. 
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4.  The  Build-Up  Affine  Scaling  Algorithm 

At  the  start  of  each  major  iteration  t ,  we  have  given  am  interior  y  =  y‘  with 
u  =  c  -  y*  A  >  0.  We  order  the  columns  by  some  criterion  such  as  (13.0)  which 
measures  the  “promise”  of  a  column  A,}  being  in  the  optimal  basis.  The  subroutine 
is  as  follows: 

choose-m-promising-col  umns : 

Reorder  indices  (l,2,...,n)  to  ,/j, ...,y„)  such  that 


(13.0) 

{jl  )  Ja »  Jm  ) 

(13.1) 

Later  we  will  show  that  this  rule  and  some  other  alternate  rules  for  choosing  the 
starting  0  =  0*  all  lead  to  finite  convergence,  see  Theorem  7.  In  practical  applications, 
the  assumption  that  every  set  of  m  columns  is  independent  is  usually  not  correct  and 
(13.1)  is  modified  to  read:  choose  0  as  the  indices  of  the  first  m  most  promising 
independent  columns.  0  stands  for  “build  up”.  During  a  major  iteration  the  initial 
set  of  indices  (13.1)  is  built  up  to  include  additional  indices.  When  0  is  used  as  a 
subscript,  it  means  that  the  indices  j  are  restricted  to  0. 

The  second  subroutine  is 
solve-starting-  basis-  (0): 

Here  0  is  the  set  of  indices  of  the  m  most  promising  columns.  Af  at  the  start  of  a 
major  iteration  is  nonsingular  and  therefore  may  serve  as  a  basis.  The  detailed  steps 
are: 

Xfi={A0)~lb]  ir  =  c,(A,)-l\  (14.0) 

r  =  argmin(x^).  (14.1) 

ii 
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The  third  subroutine  is 
solve-ellipsoid-(y * ,  (3 ): 

Here  /?  refers  to  the  initial  set  of  promising  indices  plus  those  built-up  so  far  during 
the  minor  cycles  within  a  major  iteration.  The  ellipsoid  subproblem  to  solve  is 

(£g)  maximize  yb  subject  to  y  €  £g  =  {y  :  ||(y  -  y* ) 1  [|  <  l}. 


The  subroutine  steps  to  solve  (£$ )  are: 

Dg  =  Diag(u),); 

(15.1) 

A  =  bT(AgD;>ATg)~'-t 

(15.2) 

A  =  (A6)~ 1/3  A. 

(15.3) 

The  fourth  subroutine  is 
Snd-blocking-constraint-(s) : 

At  the  start  of  a  major  iteration,  ratio  is  set  equal  to  +00  and  the  subroutine 
scans  from  y‘  in  the  direction  A  looking  for  the  first  blocking  constraint  j  =  s;  at  the 
start  of  each  minor  cycle,  ratio  is  set  equal  to  1  and  the  subroutine  then  scans  the  line 
segment  y*  to  y*  +  A  looking  for  the  first  blocking  constraint  j  =  s.  If  no  blocking 
constraint  is  encountered,  s  =  0.  The  detailed  searching  steps  are: 
s  :=  0; 
v  :=  AA; 

for  j  :=  1  to  n  do  begin 

if  ((j  &  0)  and  (v}  >  0)  and  (uj/v,  <  ratio)  then  begin 
ratio  :=  uj./w,  ;  s:=j; 

end; 
end  for. 
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The  Main  Program  of  the  build-up  affine  scaling  method  is  as  follows: 

Program; 

Input:  dual  feasible  solution  y1  and  u1  =  (c  -  y1  A)T  >  0; 

Initialize  t  :=  1  and  opt  :=  false ; 

While  ((opt  =  false)  do  begin 
choose-m-promising-columns ; 
soJve-starting- basis- (/3)  for  0  and  r; 
if  ((*,v  >  0)  and  (c  —  i tA  >  0))  then  begin 
output  (“optimum”,  x$,  xN  =  0,  *);  opt  :=  true-, 
end; 

if  ( opt  =  false)  then  begin 
ratio  :=  +oo; 

Gnd-blocking-constralnt-(s) ; 
if  s  =  0  then  begin 

output  (“error,  primal  infeasible");  opt  :=  true; 
end; 

while  s  >  0  do  begin 

f3:=0U{s}; 

solve-ellipsoid-fy* , (3)  for  A; 
ratio  :=  1; 

fin  d-  bhcking-constraint-(s) ; 
end  (end  of  a  minor  cycle); 
x)  :=  (AAj)/(u*)a  for  j  €  0; 
z*  :=  0  for  j  & 
y‘  :=  y‘  +  A;  u‘  :=  c  -  yM; 
t  ;=t  +  l; 
end  if; 

end  while  (end  of  a  major  iteration); 
end  program. 
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5.  Proof  of  Finite  Convergence  of  Build-Up  Algorithm 


Theorem  6.  Independent  of  the  rule  used  to  choose-m-promising-columns,  the  itera¬ 
tive  process  either  terminates  in  a  finite  number  of  iterations  with  the  optimal  primal 
and  dual  solutions  or  converges  to  them  in  the  limit. 

Proof  of  Theorem  6.  Let  us  assume  it  does  not  converge  in  a  finite  number  of 
iterations.  Each  major  iteration  t,  after  it  checks  and  finds  that  the  starting  basis 
is  not  optimal,  engages  in  a  number  of  minor  cycles  building  up  0*  (the  indices  of 
the  constraints  used  to  define  the  ellipsoid  £fi )  until  there  is  no  constraint  blocking 
movement  from  y*  to  y*  +  A,  the  optimal  point  of  (£*):  i.e.,  y*  +  A  is  an  interior  point 
of  (D).  The  count  of  minor  cycles  within  a  major  iteration  cannot  be  more  than  n  —  m 
because  this  is  the  number  of  remaining  constraints  yA.,  <  ci  whose  index  j  might  be 
used  to  augment  starting  0.  Therefore  minor  cycling  always  terminates  with  a  move 
from  y‘  to  y‘+1  where  the  formulae  for  this  move  are  the  analogs  of  (2.1)  to  (2.4)  of 
Dikin’s  algorithm. 


0  =  P,  D,=  Dtag(u'), 

(16.0) 

> 

11 

(16.1) 

A  =  (A6)"1/3A, 

(16.2) 

y1+i  =  y*  +  A, 

(16.3) 

x'$=D~fi'Ar,  AT;  x*=0  for  j  $  0. 

In  addition,  we  have  the  analogs  to  (4.1)  to  (4.4) 

(16.4) 

b  =  Af  x\  =  Ax* , 

(17.0) 

0<||2>,x*Ja=A6,  \\D$x\\\  =  ||Z>x‘||, 

(17.1) 
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because  x*  =  0  for  j  &  0.  The  proofs  that  1  and  the  ( Ag  D~2  ATg)~  1  exist  are  the 
same  as  those  without  the  subscript  0,  see  (3.1)  and  (3.2)  implying  that  steps  (16.1) 
and  (16.2)  are  legal.  The  proofs  that 


A b  >  0,  (17.2) 

A  =  (A b)~  1/2 A  is  an  improving  direction,  (17.3) 

y’+14-y*6=  (A4)*'a  =  \\D,z‘t\\  =  \\Dx>\\  >  0  (17.4) 

follow  the  proofs  of  (3.3)  and  (3.4).  The  analogs  to  (3.5)  and  (3.6)  are 

u;+1  =u*  -\\Dfix\\\-l{Dlx\)T,  (17.5) 

0  <  u‘+ 1  <  2u^ ,  0  <  u‘+ 1 ,  (17.6) 


where  u‘+ 1  >  0  for  j 0  is  true  because  the  minor  cycles  do  not  terminate  until  there 
is  no  blocking  constraint,  i.e.,  u‘+1  =  c,  —  yt+lA.,  >  0  for  all  j  £  0. 

Under  the  assumption  of  no  finite  termination,  we  now  show  that  Theorems  1  to 
5  of  Dikin’s  algorithm  are  true  for  the  build-up  algorithm.  However  the  reasons  why 
they  are  true  will  need  more  explanation  in  places  and  this  we  will  now  proceed  to  do. 

That  u*  z‘  —  *  0  as  t  — ►  oo,  Theorem  1,  follows  from  ( yt+1b  —  y*b)  -*0  and  (17.4). 
The  proof  of  Theorem  2  is  the  same.  It  states  that  given  any  infinite  subsequence  S0 
of  t,  there  exists  a  subsequence  5.  C  S0  such  that  (x‘ ,  y‘ )  converge  using  t  in  5.  to 
(2,y),  a  pair  of  basic  primal  and  complementary  basic  dual  feasible  solutions. 

Next  to  show  Theorem  3  for  the  build-up  algorithm  that  ( x*,y *)  converges  as 
t  -*  oo  to  a  pair  of  basic  primal  and  complementary  basic  dual  feasible  solutions, 
we  note  all  steps  of  the  proof  apply  up  to  the  very  last  step  where  relation  (3.6) 
that  u‘* 1  <  2u‘  for  t  =  tk  in  the  subsequence  Tx  is  used  to  obtain  a  contradiction. 
However  the  analog  of  (3.6)  for  the  build-up  algorithm  is  (17.6)  which  states  uj+ 1  <  2u‘ 
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provided  r  £  /?  =  /T.  We  claim  that  except  for  a  finite  subsequence  of  t  in  T,  that  r 
is  indeed  in  /3* ;  if  so  then  this  is  clearly  sufficient  to  complete  the  proof  of  Theorem 
3.  Suppose  on  the  contrary  there  is  an  infinite  subsequence  Rx  in  such  that  ,  for  t 
in  Ri,  r  not  in  Since  Rx  C  Tx  is  infinite,  its  convergence  properties  are  the  same 
as  Ti  and  therefore  x\  — ►  x*  and  \x*  \  0  because  r  is  basic  index  in  ( B,N ).  On  the 

other  hand,  x‘  —*  x\  =  0  for  t  in  Ri  because  the  algorithm  sets  all  xj  =  0  for  j  &  /?* 
which  contradicts  \x‘  |  ^  0  for  r  €  B. 

Also  requiring  explanation  is  Theorem  4  which  asserts  that  (x‘,y‘)  converge  to 
complementary  optimal  basic  primal  and  dual  solutions.  The  proof  assumes,  on  the 
contrary,  x*  is  not  feasible  and  that  there  is  some  basic  index  j  =  r  such  that  z*  — ► 
x‘  <  0  as  t  — *  oo.  To  get  a  contradiction,  relation  (11.1)  was  used;  its  analog  (17.5) 
is  applicable  provided  r  €  /?* .  However  there  can  be  at  most  a  finite  count  of  t  such 
that  basic  r  £  (3*  and  a  corresponding  count  of  x*  =  0,  because  if  the  set  S  of  r  /?* 
were  infinite,  this  would  imply  x\  -*■  2'r  =  0,  but  this  limit  is  the  same  as  that  for  all 
t,  contradicting  [x*|  ^  0.  Therefore  there  exists  a  finite  f0  such  that  for  all  t  >  t0  that 
B  £  0*  and  (17.6)  is  true  for  all  r  £  B,  and  the  rest  of  the  proof  of  the  Theorem  4  can 
now  be  applied  without  further  change. 

Finally  the  proof  of  Theorem  5  about  the  asymptotic  ratio  of  convergence  being 
p*  <  1  —  (l/m)l/a  may  be  used  without  change.  Q.E.D. 

This  completes  the  proof  of  Theorem  6  which  asserts  that  if  the  build-up  algorithm 
does  not  detect  an  optimal  basic  feasible  solution  at  start  of  some  major  iteration  t 
and  terminate,  then  it  will  converge  in  the  limit  to  such  a  solution. 

We  now  present  several  alternative  rules  that  rank  the  promise  of  column  j  being 
in  the  optimal  basis  according  to  the  ratio  (x‘)M /(«*)*'  from  high  to  low: 

Rule  1  (/*>*'')  =  (0, 1): 

(!/«*,  )>(!/«;,)>..  .>(!/«;„). 
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Rule  2  (/x,  u)  =  (1,0): 


(*r/i)>«‘/i)  >->(*•:  Vo- 

Rule  3  (fi,  u)  =  (1, 1): 

K 70  >K;Vu‘j  >...>(*5;: 7«‘J- 

Theorem  T.  If  the  starting  /?*  =  Oi  )  is  chosen  by  any  of  the  above  rules, 

the  build-up  algorithm  will  terminate  in  a  finite  number  of  iterations  with  the  optimal 
primal  and  dual  solutions. 

Proof  of  Theorem  7.  Let  the  optimal  partition  be  ( B,N ).  Assume  on  the  contrary 
that  the  particular  rule  chosen  does  not  lead  to  finite  termination,  then  by  Theorem 
6  as  t  -*  oo,  (xg,!")  converge  to  optimal  (iB  >  0,£*N  =0);  and  (u^.u^)  converge 
to  (u^  =  0,u^  >0).  Let 

«o  =  (1/2)  min(2*€B  ,u*€Ar ). 

There  exists  a  t0  such  that  for  all  t  >  t0: 


u‘  <  Co 

i€B 

and 

>  c0  j  €  N. 

X*  >  Co 

j€B 

and 

<  c0  j  €  N. 

x‘/u‘  >  1 

}€B 

and 

< 

/«‘  <i  j  e  N. 

Therefore,  contrary  to  the  assumption  of  convergence  in  an  infinite  number  of  iter¬ 
ations,  we  see  that  under  any  one  of  the  three  rules,  the  basis  associated  with  the 
starting  0*  for  all  t  >  t0  is  optimal  and  the  algorithm  will  detect  this  fact  by  iteration 
t<t0.  Q.E.D. 
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6.  Number  of  Minor  Cycles 

The  count  h*  >  m,  the  number  of  indices  in  0  =  0*  build  up  by  the  end  of  major 
iteration  t,  is  not  fixed  and  can  vary  up  and  down  from  one  iteration  to  the  next.  We 
expect  in  early  iterations  that  h*  will  be  somewhat  large,  but  as  t  increases  h*  will 
likely  be  close  to  m.  Also,  one  may  allow  several  columns  to  enter  A0  simultaneously, 
instead  of  one  at  a  time,  and  allow  some  columns  deleted  from  A0  in  order  to  keep 
the  size  of  A0  under  control.  In  the  following,  we  show,  at  least  heuristically,  that  the 
size  of  A0  might  be  controlled  to  no  more  than  2m  for  problems  with  n  >  2m.  This 
is  based  on  the  following  theorem: 

Theorem  8.  Let  ft  =  {y  €  Rm  :  c  —  yA  >  0}  where  A  €  Rm  x  "  and  n  =  2m  +  1,  and 
let  y°  be  any  interior  point  in  fi.  Also,  let  A0  denote  an  m  x  (2m)  submatrix  of  A. 
Then,  there  exists  a  particular  A0  such  that 

={y:||(y-y°M/,z?;l||<i>cn 

where  D0  =  Diag (c0  -  y°A0). 

Proof.  We  know  that 

Sf  C{y:c0  -  yA0  >  0}. 

Therefore,  we  only  need  to  show  that 

S0  C  {y  :  c,  -  yA,,  >  0}  for  s  £  0. 

Let  a j  A.j ,  D  =  Diag(c  -  y°  A)  and  let  a  be  the  vector  whose  j- th  component  is 

<7,  >0, 

which  are  diagonal  components  of  the  projection  matrix 

D~ 1  At  (AD~ 3  At  )  ” 1  AD~ 1 . 
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The  trace  of  the  projection  matrix  satisfies 


3m  +  1 

]C  =  m- 

3  =  1 

Therefore,  there  exists  at  least  one  j  =  s  such  that  a,  <  1/2.  Define  0  to  be  the  2m 
indices  which  excludes  s.  We  show  £g  C  (y  :  c,  —  ya,  >  0},  i.e.,  constraint  s  does  not 
block  C6  in  any  direction.  As  shown  in  Reference  [12j,  it  suffices  to  prove  that 

&.=a,(A$D;'AT$)-la./Dl<  1. 

This  is  because  the  following  argument.  The  maximal  value  of  the  problem: 

maximize  (y  —  y°  )a,  subject  to  y  €  £0 


is  exactly 

(y  -  V°)a.  =  yJaMeD-'ATy'a, 
where  y  denotes  the  maximum.  Thus,  if 


c.  ~y°a.  =  D„  >  \Ja.{A8 D~2A*)~la,  =  (y-y°)a,, 


then 


c,  >  ya.  >  yo.  for  all  y  €  £g . 


Note  that 

(AD-*At)-‘  =(A,D;'AT,+a.,l /&„)-' 

=  (a,d;’aJ)-‘ -(a,d;’aZ)-‘<i.«?(a,d;’aZ)-'/(dUi  +  9.)).  (is.i) 

Multiplying  (18.1)  by  af  and  a,  from  the  left  and  right,  and  then  dividing  it  by  D]t , 
we  have 

a.  =  9.-97J{\+9.) 
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which  yields 


O.  =  o'. /(I  -  o,). 

Thus,  a,  <1/2  indicates  that 

0,  <  1 

This  completes  the  proof  of  Theorem  8.  Q.E.D. 

Theorem  8  states  that  the  ellipsoid  inscribed  in  polytope  0  with  2m  +  1  con¬ 
straints  can  be  constructed  using  at  most  2m  constraints,  which  suggests  but  does  not 
establish  that  h‘  will  be  at  most  2m.  This  helps  when  faced  by  an  n  >  2m  LP  problem 
because  if  the  number  of  components  of  0  becomes  greater  than  2m  the  calculation  of 
A9D;2At,  will  be  a  big  burden  as  it  is  in  conventional  interior- point  algorithms.  Note 
that  the  build-up  method  can  also  be  used  for  optimization  problems  where  linear 
constraints  are  generated  during  the  course  of  the  algorithm. 

A  rank-one  updating  method  should  be  used  to  efficiently  refactorize 


M.VO" 

and  recalculate  A  after  Afi  is  augmented  by  a  blocking  dual  constraint  s.  This  efficiency 
can  be  achieved  using  any  one  of  several  techniques  (see,  for  example,  Gill  et  al.  [6] 
and  Ng  [9]).  Also,  at  the  beginning  of  each  major  iteration  we  need  to  invert  a  basis 
typically  in  factorized  form.  If  the  basis  differs  from  the  one  at  the  beginning  of  the 
previous  major  iteration  by  only  few  columns,  we  can  apply  a  few  rank-one  updates 
to  obtain  its  inverse  in  factorized  form  as  well. 
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Abstract 

We  propose  a  build-up  interior  method  for  solving  an  m  equation  n  variable  linear 
program  which  has  the  same  convergence  properties  as  their  well  known  analogues 
in  dual  affine  and  projective  forms  but  requires  less  computational  effort.  The  algo¬ 
rithm  has  three  forms,  an  affine  scaling  form,  a  projective  scaling  form,  and  an  exact 
form  (that  uses  pivot  steps).  In  this  paper,  we  present  the  first  of  these.  It  differs 
from  Dikin’s  algorithm  of  dual  affine  form  in  that  the  ellipsoid  chosen  to  generate  the 
improving  direction  A  in  dual  space  is  constructed  from  only  a  subset  of  the  dual 
constraints. 

At  the  start  of  each  major  iteration  t,  we  are  given  an  interior  iterate  y‘.  A 
selection  of  m  dual  constraints  is  made  using  an  “order-columns”  rule  as  to  which 
constraints  show  “the  most  promise”  of  being  tight  in  the  optimal  dual  solution.  An 
ellipsoid  centered  at  y‘  is  then  inscribed  in  convex  region  defined  by  these  promising 
constraints  and  an  improving  direction  A  computed  that  points  to  the  optimal  point 
y1  +  A  on  the  ellipsoid  boundary.  Minor  cycling  within  a  major  iteration  is  then 
started. 

During  a  minor  cycle,  the  constraints  selected  to  define  the  ellipsoid  centered  at  y‘ 
is  built  up  to  include  the  constraint  (whenever  there  is  one)  that  first  blocks  feasible 
movement  from  y‘  to  y‘  +  A.  If  one  blocks,  it  is  used  to  augment  the  set  of  promising 
constraints  and  the  ellipsoid  is  revised;  the  improving  direction  A  is  recomputed  by 
means  of  a  rank-one  update,  and  the  minor  cycle  repeated  until  none  blocks  movement 
from  y‘  to  y*  +  A.  When  none  blocks,  the  minor  cycling  ends.  yt+1  =  y‘  -f-  A  initiates 
the  next  major  iteration.  Major  iterations  stop  when  an  optimum  solution  is  reached. 
We  prove  this  will  occur  in  a  finite  number  of  iterations. 
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